With the use of the Internet, email and related electronic services, communications software has been increasingly called upon to handle data in a variety of formats. While the barriers to simple communications have been removed from many hardware implementations, the problem of software being unable to display text in different languages remains.
For instance, a person browsing the World Wide Web may wish to input a search string in their native language. Some Web pages or search engines will simply accept that string in the form in which it was input, but not process the spelling, syntax or character set in native form. The search engine then performs a search as though the search were in English, usually resulting in no hits. Other Web pages may allow user to manually specify the desired language for browsing and searching.
To display multilingual data correctly on the output side when a document is located and ready to be outputted on a computer screen, sent to a printer or otherwise using the right font is a common problem that developers encounter when writing international applications, including Web applications.
In the pursuit of better and more uniform multilingual documents, the International Standards Organization (ISO) and other bodies have developed a universal character set standard referred to as Unicode, Version 2.0 of which was released in 1996. The current Unicode standard is a 16-bit protocol encoding 25 different scripts as well as at least 38,885 separate characters. Scripts are in general higher-level collections of related characters from a character set which may be assembled for use in one or more languages.
Ultimately, displaying Unicode data becomes a problem of dynamically selecting the closest font available on a system to best express a subject document. The commercial TrueType™ and OpenType™ font sets can only output the appropriate glyphs or symbols for a subset of Unicode ranges. It is therefore necessary to generate the right character set (charset) font flag information, that is, to identify the font best matching the data to be displayed when creating the graphical font object for display or printing.
Unfortunately, documents often originate from unknown sources and unknown languages and even if the original text is encoded in Unicode, it can be difficult to identify in which character set or Unicode target range the content has been encoded with. Once the character set is identified, it can be mapped to the corresponding font flag information used to create the logical font under a given operating system. For example, text that can be encoded as Code page 874 would be mapped to THAI_CHARSET in Microsoft™ Windows™.
The display of Unicode data consequently relates to a process of selecting a font and using whatever system application programming interface (API) is available to output the text, a process which becomes even more complicated when the text is in multiple languages. Because the selected font might not be able to render all the characters from all the different character sets used in multipart, multilanguage documents, that type of data must be broken into different textual segments that use the same character set and display them separately using the appropriate font.
Therefore, among other technical complexities, the software developer must solve at least two fundamental problems when trying to accommodate multilingual output. The first is to determine which character set or Unicode ranges the text has been encoded with. The second, to choose the font that will be able to render the characters most correctly. Other problems exist, including the selection of most appropriate fonts for developing printed output.